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Introduction: Context of the Project. A major barrier to completion of the 
undergraduate Engineering degree is completion of two courses in Mechanical Engineering, 
Statics and Dynamics. In Statics, one requirement is learning to sol ve freebody diagram 
problems. In this task, the student is presented a concrete situation l ik e that in Figure EX1, 
showing one or more objects along with the forces acting on the system. The first job is to 
construct an abstract drawing of the objects and forces, which then allows writing equations for 
analysis of the forces at work in the system. Constructing the freebody diagram requires a 
perceptual analysis of the original problem, which can pose a variety of challenges, such as 
deciding which elements to combine and which to separate, and how to represent various forces. 
Most students have previously encountered similar tasks in high school physics, but these 
problems were generally simple sketches, such as a set of blocks and cables, requiring less 
perceptual work than the situation in Figure EX1. 




Figure EX1. Illustration of a statics problem in Mechanical Engineering (a), and (b) a freebody diagram 
constructed to represent the problem (b). Part (a) from Meriam & Kraige, 2007. 

Students are taught to solve freebody problems through lectures and textbooks, the former 
including one or more worked examples by the instructor, and the latter providing both worked examples 
and homework assignments (cf. Atkinson, Derry, Renkl, & Wortham, 2000, for a comprehensive review 
of the concept of worked examples). These learning resources vary in the level of detail about solution 
steps, and seldom allow dynamic interactions with students, although discussion sections may include 
such activities. 

This study explores data from an investigation of individual differences among students in the 
early stages of acquiring the freebody concept. Newton 's Pen (Lee, de Silva, Peterson, Calfee, & 
Stahovich, 2008) reported findings from a pilot study in which Engineering freshmen completed a 
computer-based pen (CBP) tutorial designed to assist them in acquiring the free body diagram concept. 
The students showed substantial improvements in performance over the course of an hour-long session; 



few students could solve the pretest problem at the outset, while everyone handled the same problem 
efficiently and effectively following the tutorial. 

The paper presents a secondary analysis of Newton ’s Pen exploring individual differences in how 
students approached the problem. Some students worked out the problem, proceeding hastily and making 
numerous mistakes in an apparent effort to complete the assignment quickly and by any means possible. 
Other students worked through the problems, taking time to study instructions, keeping multiple pages in 
view during the task, reflecting on their mistakes, and in general displaying characteristics typical of 
expert performance. In this paper we propose the concept of an expert stance to describe the second 
group, and then consider ways in which a CBP platform might promote this stance during learning of the 
freebody task. 

The Computer-based Pen Platform. CBPs such as those developed by the Anoto Group provide 
a novel platform for the writer, where “writing” may encompass activities ranging from prose to 
scribbles, from outlines to semantic webs, sketches, cartoons, and schematic drawings — virtually 
anything that can be captured with pen and paper. Advanced CBP platforms such as Livescribe’s Pulse 
(Figure 2LS) provide a variety of supportive functions, including storage of audio and graphic material to 
support note-taking and pencasting activities, and a docking capacity that allows exchange of information 
between the pen and a digital computer. 




Figure 2LS. The Livescribe Pulse, a Computer-based Pen Platform. 

CBPs have both advantages and limitations compared with platforms such as pencil-paper, 
keyboard-mouse, tablet PCs and I-PADs, and PDAs. The pros and cons hinge on factors such as cost, 
flexibility, convenience, and power, among other considerations. Oviatt (in press; also Oviatt & Cohen, 
2010, Oviatt, 2006, and van Schaack, 2009) has reviewed these issues from conceptual and empirical 
perspectives, and has conducted several studies with high school students engaged in solving science and 
mathematics problems. This research revealed that digital pen and paper interfaces entail affordances that 
stimulate higher rates of nonlinguistic communication (e.g., diagramming, symbols, and numbers), 
ideational fluency, problem solving, and retention of domain content, compared with other graphical 
interfaces such as PCs and pencil-paper platforms. They also appear to lower cognitive load, provide 
more effective support for low-performing students, and reduce expansion of the performance gaps 
among student groups. As an example of the functional advantages of CBPs, students can spread out an 
array of pages while working on a problem, allowing continuous spatial referencing (Jang, Schunn, & 
Nokes, 2011). A full treatment of these issues falls outside the purpose of this paper, which will focus on 
the role of worked examples during learning (Atkinson, et al., 2000), and the educational potential of 
CBPs in this setting. 

Over the past several years, we have explored the application of the CBP platform for the 
development of tutorial situations to enhance performance on and understanding of homework exercises 
assigned to beginning engineering students in the statics course. Mechanics courses serve as a significant 
gateway in the path toward completion of the Engineering major. Homework assignments are important 
both in their own right and in preparing students for examinations. Engineering textbooks contain 
substantial numbers of homework exercises, along with worked examples. Following the release of a new 
edition for any of the standard textbooks, however, solutions to the homework exercise sets quickly 



become available on the internet, which students can reproduce in homework assignments with minimal 
effort, with little or no reflection, and - in all likelihood - with limited impact on learning and 
understanding. 

Novice efforts to work through a statics problem without assistance can be difficult and 
frustrating, which explains why students turn to short cuts. We have employed the CBP platform to 
support students during completion of freebody diagrams in several experiments, some aimed toward the 
validation of assessment techniques, and others in which tutorial techniques were the focus. This report 
uses data from sixteen participants in a pilot project, Newton ’s Pen, where the CBP platform provided 
tutorial support (Lee, de Silva, Peterson, Calfee, & Stahovich, 2007). The research design employed a 
within-subjects design in which participants completed a pretest (the “Ring” problem, Figure 3RING) 
followed by a worked-example tutorial in which they completed a freebody diagram problem while a 
worked example was available for reference. Two problems were developed for the tutorial and transfer- 
test phases, the Inclined Plane (IP) and Friction (FR) problems. For half the participants, the IP problem 
served for the tutorial and the FR problem for the transfer test; for the other participants, the order was 
reversed. Participants first attempted the Ring Pretest, which presented a considerable challenge for most 
of them. Analysis of performance on the assessment series (pretest, posttest, re-pretest) reported in Lee, et 
al. (2007) showed that all participants subsequently attained high levels of success and expressed 
satisfaction following the Tutorial. 




Figure 3RING. The "Ring," which served as a pretest for Newton's Pen. 

This paper explores performance during the Tutorial, a “worked example” scenario, during which 
a scaffolded support system was available to participants using the CBP platform for implementation. The 
Tutorial began with a segment on using the pen to “compose” the features required during analysis of a 
freebody diagram: drawing force arrows and angles, labeling various elements, and so on. After this 
segment, participants were presented the freebody problem and a model of a worked example (Figure 
4WE illustrates the tutorial system for the Ring problem). Following the worked example, students were 
presented a page with a box where the student was to sketch the freebody diagram, and a second page for 
writing the force equations. This paper focuses on analyses of behavior during the freebody sketches; 
performance on the equations will not be covered here. 
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Figure 4WE. Newton’s Pen Tutorial Layout for Ring Pretest. 

The tutorial guided students through several activities. They first constructed an X-Y coordinate 
system as a reference, next drew the freebody, and then added the force vectors. Following each stroke, 
the CBP responded aurally with feedback about the correctness of the stroke, and offered guidance about 
how the participant should next proceed. If the participant was unclear about the feedback, a Help icon 
was also available with three levels of assistance: Level loffered general guidance, Level 2 a more 
focused hint, and Level 3 gave the exact answer. The CBP prototype used for the pilot study had 
limitations; in particular, feedback to responses was rather slow. Although participants were advised to 
wait for feedback, many were impatient and moved ahead before receiving feedback to a particular 
response, which led to complications during the first part of the Tutorial. Note that the Worked Example 
was available for review throughout the reproduction process, a feature of the tutorial that was possible 
with the CBP platform, because unlike a laptop computer, the worksheets could be spread out on the 
table. Notice also that a participant could complete the Tutorial by simply copying the worked-example 
model. Finally, a participant could jump to Level 3 of the Help icon to obtain detailed help, allowing 
completion of the transfer tasks without doing any work. 

In setting the stage for the following analyses, we should remark that every participant appeared 
to attempt more than the “copy” defaults mentioned above. Participants were volunteers from a beginning 
Physics class, had just completed an introductory lecture on freebody diagrams, and had received 
homework assignments that were due the following week. It is reasonable to assume that they were 
strongly motivated to learn something during the tutorial session. All students seemed engaged in doing 



more than simply reproducing the worked-example model, and in general seemed to be trying to 
understand the freebody process. In reviewing tutorials during test scoring, however, we noticed 
substantial individual differences among participants in how they approached the task. Some individuals 
tapped the Help icon much more than others, appearing in some instances to be relying on the third and 
most specific Help level in order to reproduce the worked example with limited effort. Other individuals 
were remarkable because of the extended time they spent studying the worked-example before turning the 
page. They also tended to spread out the pages rather than placing them into a stack, a behavior typical of 
the first group. 

The Newton ’s Pen data set provides a rich context for generating hypotheses about student 
strategies during a WE task. The WE literature has explored various techniques for optimizing the value 
of an example; e.g., by eliminating irrelevant distracters that might increase cognitive load and impair 
performance (cf. Rossow, 2005; Atkinson, Renkl, & Merrill, 2003, for comprehensive reviews of this 
literature). Newton’s Pen, because of the fine-grained nature of the data, allows detailed exploration of 
contrasts between what we have labeled “worked-out” versus “worked through” responses. Atkinson, et 
al. (2003) used “worked out” to describe a prompt paradigm that they employed in place of the “figure it 
out yourself’ strategy employed in many WE investigations. Meier, Reinhard, Carter, & Brooks (2008) 
embedded computer-supported “elaborated worked-example modeling” in scenarios for students during a 
forensic science class. They found that these augmentations enhanced learning for novices. 

In a major conceptual move, Meier, et al. (2008) also introduced the novice-expert contrast as 
background for conceptualizing the approaches used by more or less successful students during WE 
exercises. Numerous investigations have revealed distinctions between novices and experts, yielding a 
profile of expert performance characterized by the features shown in Table lExpNov. A consistent 
feature of expert problem-solving, emphasized by boldface in the table, is the tendency to allocate a 
significant amount of time to examination of the situation. Before beginning work on a problem, experts 
tend to study the problem very carefully. They also show give evidence of self-reflection, of monitoring 
and reviewing their performance, especially when they make mistakes. 

Table lExpNov. Characteristics of Expert Performance (After Chi & Glaser, 1988) 

• Excel in a specific domain 

• Process information in large units 

• Performance is fast and smooth 

• Hold more information in memory 

• Represent problems at a deeper level 

• Spend more time analyzing the problem 

• Are better monitors of their performance 



The novice-expert distinction seems to us to have significant parallels to the work-out vs. work- 
through contrast that we have adopted to describe variations in the Tutorial activities of Newton ’s Pen 
participants. A qualitative analysis of the data set generated several indicators that we plan to use in 
further investigations of other data sets now under analysis. The notion of exploring individual 
differences in WE performance during completion of a tutorial package appears to be innovative in its 
own right; Atkinson, Derry, Renkl, and Wortham (2000) mention “individual differences” in their review 
of WE papers, but their emphasis is on self-explanation findings rather than individual differences. 





Table EXCL displays the Excel template used to compare selected performance features during 
the Tutorial. The task was divided into several segments, for each of which beginning and ending times 
are noted. Comments were written for each segment based on information from the videos and the sketch 
performance recorded by the Pen. The Table provides brief samples from two cases, the first typical of 
the WORK-OUT profile and the second of the WORK-THRU profile. 



Table 2EXCL. Spreadsheet template for qualitative analysis of informant profiles during Worked- 
Example Tutorial Exercise. 
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Response time provided a revealing indicator when segmented as shown in the spreadsheet. 
Figure 5TIMES shows the proportion of total time taken by two informants in each of the freebody- 
sketch segments, normalized by total sketch time. The first time metric was the length of the initial 
examination. Some participants took 30 seconds or less to study the worked-example model before 
beginning to draw/copy the model. Others took three minutes or more to complete the examination. 
Another set of quantitative metrics were the times for completing various segments of the freebody 
diagram. As indicated in Figure 5TIMES, Work-THRU participants tended to proceed quite smoothly in 
sketching a segment once they started to draw, while Work-OUT learners were more likely to proceed in 
a jerky, start-stop fashion, which meant that they took longer to complete the sketch elements. 




Figure 5TIMES. Proportion of freebody sketch time taken by each of two informants, normalized 
on total sketch time. 

Participants were encouraged to think aloud during all of the tasks. Only a few complied with this 
request, however, and so we have limited evidence about what they might have been thinking. In similar 
fashion, most participants seldom asked questions of the examiner, perhaps because of the implied 
assumption that the Pen was supposed to do the talking. On the other hand, several participants offered 
spontaneous remarks during the task, samples of which are provided in Table 2EXCL. Work-THRU 
respondents were more likely to comment on the task, while Work-OUT learners typically offered 
personal reactions to the situation. Interestingly, the informants labeled as THRU and OUT did not differ 
greatly in the total amount of time they spent working on the problems, and virtually everyone succeeded 
on the Post and Post-Post problems. The observed differences tended to be in the way that work time was 
distributed, and in the number of mistakes along the way. 

Another contrast of potential significance appears in responses to errors and feedback. Some 
participants quickly tapped the Help icon once they discovered how it worked. They often repeated a 
previous response following a mistake, generally to little avail. These participants tended to react in 
scattershot fashion when confronted with negative feedback; they seldom paused to reflect on various 
explanations and options available to them, but rather continued to draw strokes in a haphazard manner. 

Table 3THRU-OUT summarizes the contrasts drawn above. The data set is small, and serves 
descriptive rather than inferential purposes. Nonetheless, the patterns are sufficiently consistent that we 
think that they warrant attention, and that the novice-expert distinction may have conceptual and practical 
value in thinking about the THRU-OUT contrast. 



Table 3THRU-OUT. Summary of Contrasts between Work-OUT and Work-Thru profile 


Work THRU 


Work OUT 


• Spends significant time studying 


• Skims quickly through instruction 


instructions and reading the problem 


• Goes to work quickly on a problem 


• Refers to instructions and previous work 


• Uses hints/help frequently to move past 


frequently during the task 


barriers 


• Uses hints and help selectively 


• Learns little from mistakes, which tend 


• Studies mistakes carefully; seldom 


to be repeated 


repeats 





We propose that the Work-THRU profile may signal progress by a novice student toward 
development of expertise, a situation that we have labeled as the adoption of an expert stance. Our 
hypothesis is that certain performance characteristics play a critical role in making progress toward 
expertise, including the two bold-faced elements highlighted at the bottom of Table lExpNov. These 
characteristics call to mind metacognitive activities in which a learner stands apart from ongoing 
performance, often “thinking aloud.” An important correlate of metacognitive activities is the presence of 
reflective and self-regulatory actions. As Oviatt (in press, Chapter 11) notes, individuals differ in their 
tendencies toward a reflective versus an impulsive style, but these tendencies also depend on the activity 
context. For example, multi-tasking environments seem to foster impulsive reactions, while tasks that call 
for written activities lead to more reflective responses. Oviatt does not report on studies of instruction 
explicitly designed to foster a reflective, metacognitive posture, but offers a number of suggestions that 
seem useful starting points for such endeavors. 

Pathways to expertise. An idea of potential importance springing from these analyses is the 
notion that employing the Work-THRU approach may support the movement from novice to expert. From 
this reasoning, a significant instructional question centers on the design of activities that might promote a 
Work-Through strategy, that could provide students with feedback not only on content -relevant aspects of 
performance, but also on performance characteristics that conform to or differ from the Work-THRU 
profile. The CBP platform offers a variety of opportunities for exploring this hypothesis, and for 
designing tutorial programs that reinforce significant features of the Work-THRU profile. 

An important next move in our work is accordingly an attempt to formulate and evaluate 
conditions that foster a “worked through” approach to worked-example opportunities. For a variety of 
reasons, copying a model (the ultimate OUT approach) has considerable appeal to students; it is easy, 
fast, and requires little effort, mental or otherwise. An even simpler approach is to glance at the model 
and move on. While this approach offers an easy approach to homework assignments, it probably 
provides limited support for transfer to situations with more difficult problems, or in more closely 
monitored settings such as in-class examinations. 

The expert stance emerges in learners who, for whatever reason, approach a domain with a 
reflective, metacognitive attitude, which on the surface may seem slow and tedious, bringing to mind the 
tale of the “hare and tortoise,” but which may be critical for moving toward expertise. The prevailing 
tendency seems to be for students to try to emulate fluent expert performance (the teacher, for example), 
but such efforts may turn out to be disadvantageous for novice learning. The attainment of speeded 





behavior is likely to require an apprenticeship during which the essential building blocks for dealing with 
a domain are explored gradually and with attention to mistakes and dead ends (one is reminded of the 
tutorials in The king’s speech). Chi (2006/46, Table 2.1) presents levels of a proficiency scale (Novice, 
Initiate, Apprentice, Journeyman, Expert, Master) that suggest a series of qualitatively distinctive stages 
in the progression from novice to expert and beyond, an idea that would seem to warrant further thought 
and empirical study. 

Instruction that promotes an expert stance may entail the application of adaptive expertise 
(Hatano & Imagaki, 1986), in which accomplished performance is slowed down and analyzed, as though 
the situation is being viewed simultaneously through both expert and novice lenses. Adaptive expertise 
may be a critical element undergirding the expert’s capacity to explain the problem-solving process in a 
particular situation (Hmelo-Silver & Pfeffer, 2003). The tutorial principles employed in the design of 
Newton ’s Pen were relatively simple: provide immediate feedback along with graduated levels of 
assistance. Augmentations to promote an expert stance would include attention to both time and space: 
“(1) Take time to study the problem, to think about what you are doing and why, and (2) be sure to keep 
all of the information that you need in view, and to regularly survey the situation.” These are relatively 
simple principles, but implementing them in conjunction with feedback and graduated help will require 
further design work, both conceptual and empirical. The extensive literature on cognitive load theory, 
which has served as the basis for much of the empirical information on worked examples, also needs to be 
added to this mix (cf. Sweller, Ayres, & Kalyuga, 2011; Plass, Moreno, & Brunken, 2010, for recent 
reviews). Cognitive load theory proposes that task performance, particularly for individuals in the early 
stages of learning a complex task, can become chaotic if contextual circumstances overload the mental 
resources available for the task. The learner then spends so much energy managing resources that little 
remains to handle the substance of the problem. Oviatt (in press) emphasizes the advantages of digital 
pens for reducing extraneous interface load while increasing germane load in tasks such as constructing 
diagrams while working on STEM problems. 

In summary, our thinking about the development of expertise has led us to consider a multiphase 
pathway toward the acquisition of expertise: 

• Adopting a metacognitive stance, with more attention to analysis of the problem, reflection on 
mistakes, and ongoing review of performance, including “looking at all of the pieces.” 

• Seeking assistance in more fully understanding how to approach a domain; asking for help; 
thinking about the implications of various courses of action. 

• Constructing a mental model of the domain, probably in segments or chunks (Simon, 1996), 
interactively, and shaped by experiences with a variety of different problems in the domain. 

• Repeated practice and feedback, refinement and automatization of templates, for both structures 
and processes. 

• Leading eventually to speed and internalization of learning. 

Expertise, in this formulation, is not a gradual progression across all facets of accomplishment, but 
proceeds through advances in selected facets, with reflection and self-monitoring playing a critical role in 
the early stages, and during later stages when new situations require rethinking the problem context. 




Concluding Thoughts 

Our studies of CBP-based tutorials were initially motivated by questions about the effectiveness 
of different platforms. Several features of smart pens appear advantageous when compared with 
alternatives such as tablet PCs and IPads. Paper-pencil technology is commonplace from the early ages 
onward, which may be why learners tend to perform better with smart pens on a variety of cognitive 
metrics (Oviatt & Cohen, 2010; Stahovich et al., ASEE, 201 1). The smart-pen platform allows students to 
display multiple sheets of paper for ongoing reference during learning. In Newton ’s Pen, learners had to 
manage a collection of different pages: instructions for using the Pen, information about the problem and 
the tutorial model, their own work on the FBD sketch, and so on. Many students made limited use of 
these resources; they placed successive sheets of paper on top of each other or otherwise disregarded 
them. One notable characteristic of Work-THRU learners was their ongoing reference to these materials, 
suggesting that a pen-paper environment which may have been an important feature of their problem- 
solving strategy (Jang, Schunn, & Nokes, 201 1). We are not suggesting that any given platform be given 
priority over others, but rather see advantages to instructional design that builds upon a seamless 
collection of devices, including smart pens, but also incorporating personal and tablet PCs, white boards, 
and so on. 

A second line of reasoning from the current study centers on the tutorial design. Virtually every 
student showed considerable progress during the hour-long tutorial. Some reasons for the change are the 
usual suspects: individualized practice with immediate feedback, opportunities for constructivist 
engagement with the task, and the innovative platform. They were also motivated by the upcoming 
homework assignment. Other reasons may spring from new twists on old favorites: the availability of 
dynamic worked examples, and ready access to hints and helps. 

The emergence of the expert stance during the performance of some learners raises several other 
questions about effective and efficient instructional design. Some issues are rather general, others specific 
to the freebody diagram situation. For example, it may make sense to incorporate procedures that promote 
reflective thought (e.g., “you should take time to study the instructions and the problem”) and to 
discourage “quicky” responses, as when a student moves rapidly from one page to another. Individual 
tutorials could also offer guidance about strategies for responding to errors of various types. The difficulty 
of sketching a freebody diagram depends on the complexity of the problem, which may not be 
immediately obvious to the novice. For example, the two problems in Figure 6JAWS may appear similar 
on the surface, but they call for quite different solution strategies. Examination of learners’ freebody 
sketches can quickly reveal whether a learner perceives these conceptual differences, information that can 
then serve for tutorial guidance. 





Figure 6JAWS: Devices with superficial similarity but conceptual differences. On the left, DF and EF are two-force 
members. On the right, there are no two-force members. (Figures from Meriam & Kraige, 2007].) 



Our final thoughts revolve around implications for promoting the adoption of an expert stance in 
this setting. Developmental considerations are seldom mentioned in the expert-novice literature, but the 
attainment of expertise in a well-defined domain seldom emerges prior to secondary education, and more 
typically after an individual begins to pursue a career or a craft. The decision to pursue a degree/career in 
engineering or other professions and crafts often entails a commitment to the attainment of expertise, 
which means moving through a series of stages from apprentice to master (Chi, 2006). Early in this path, 
students may not be fully aware of the changed situation, but they become gradually aware that schooling 
is “serious,” with the attendant need to adopt new attitudes and habits, which we suggest should include 
adopting an expert stance toward learning. Above we discussed some practical aspects of such a stance: 
taking time to reflect on problems, learning from mistakes, and so on. From a broader perspective, the 
expert stance can be seen as one element in a package designed to promote success by all students who 
decide to pursue careers in engineering and other high-stress, low-retention fields. Developing such a 
package may appear to be a daunting task, but we suggest that it is more approachable when decomposed 
into “bite-size” segments, for which Newton ’s Pen may serve as a workable model. 
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